} 



DOCUMENT RESUME 



ED 062 705 



EA 004 256 



AUTHOR 

TITLE 

INSTITUTION 

PUB DATE 
NOTE 



Hegedus, Rita 

Educational Auditing System. 

Delaware State Dept, of Public Instruction, Dover. 
Div. of Research, Planning, and Evaluation. 

Dec 7 1 
13p. 



EDRS PRICE MF-$0.65 HC-$3.29 

DESCRIPTORS ^Educational Accountability; Educational Improvement 

♦Evaluation Metiiods; *Guidelines; ♦Program 
Evaluation; ♦State Departments of Education; State 
Supervisors 

ABS*IK ACT 

This position paper describes the philosophy, 
rationale, and activities of the Evaluation Section of the Delaware 
Department of Public Instruction for the purpose cf establishing 
operational guidelines. Under a new system established in Delaware, 
the role of the evaluation supervisor at the State level has become 
one of independent, objective observer of locally operated 
educational projects or programs. The information obtained by the 
evaluators from the audit is then provided to the decisionmakers 
responsible for the project. (Author/JF) 






o 

ERIC 










i 



STATE OF 




U.S. DEPARTMENT OF HEALTH, 
EDUCATIONS WELFARE 
OFFICE OF EDUCATION 
THIS DOCUMENT HAS BEEN REPRO* 
DUCED EXACTLY AS RECEIVED FROM 
THE PERSON OR ORGANIZATION ORIG^ 
INATING IT. POINTS OF VIEW OR OPIN- 
IONS STATED DO NOT NECESSARILY 
REPRESENT OFFICIAL OFFICE OF EDU 
CATION POSITION OR POLICY 



DEPARTMENT OF PUBLIC INSTRUCTION 

DOVER. DELAWARE 



Dr. Kenneth C. Madden Dr. Howard E. Row 

Superintendent /Assistant Superintendent 

Auxiliary Services 



EDUCATIONAL AUDITING SYSTEM 



December 1971 



Dr. Wilmer E. Wise, Director 

Planning, Research, and Evaluation Division 

Dr. Rita Hegedus , Supervisor 
Evaluation Section 



o 

ERIC 



i 






PREFACE 



This position paper represents a new approach toward 
the role performed by Supervisors of Evaluation in the Depart- 
ment of Public Instruction. The formulation of this new 
role is, in part, a response to the current demand for 
accountability in education. While consistent with the 
evolving USOE definition of an educational program auditor, 
the role of evaluation has been tailored to fit the edu- 
cational context in Delaware. 

The philosophy, rationale, and activities of the 
Evaluation section are sufficiently described herein to 
establish operational guidelines. Essentially, this 
system causes evaluators at the State level to be inde- 
pendent, objective observers of locally operated educational 
projects or programs. The State Evaluators or educational 
program auditors will provide the information obtained 
from the audit to the decision-makers who are responsible 
for the project. It is hoped that the result of the audit 
procedure will be better educational programs. This position 
paper was developed by Dr. Rita Hegedus. 



Wilmer E. Wise, Director 

Planning, Research, and Evaluation Division 
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INTRODUCTION 



Education in America is no longer accepted as an 

unquestioned good. The demand for better schools has caused 

an impact on several levels of the educational community. 

Local community-action groups have, perhaps, been most 

vocal in their attack on the schools, but they are not the 

only segment of society that is concerned. The increasing 

failures to pass school bond referenda and operating refer,- 

enda as well as the crisis of cities forcing legislators to 

reappraise spending priorities have resulted in less money 

for schools. Delaware's Governor Russell W. Peterson, in a 

nationwide speech, expressed the sentiments of many in both 

the public and private sectors, when he said 

I am for commitment to Education, but not the 
blind faith of some professional educators vrho 
say just give us more dollars, let us alone and 
we'll get the job done. We have been doing 
pretty much that and the evidence is pretty 
conclus ive--the job is not getting done for 
many of the children in our society. (5) 

The problem for educators, then, is how to best 

address themselves to this growing lack of faith. On^ way to 

respond is to make the demand for accountability work for 

the schools. By definition, "accountability relates to 

the efficient and effective allocation of resources, including 

time, money, and personnel." (8) Thus, accountability at 

the state level can take a variety of forms from merit rating 

of employees to the establishment of a statewide testing 

The Delaware Educational Accountability System (DBAS) 
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which is being implemented by the Research Section of the 
Planning, Research, and Evaluation Division is, in fact, a 
major response of the Department of Public Instruction to 
the need for accountability in the schools. (7) 

The Evaluation Section of the Planning, Research, 
and Evaluation Division has formulated a position for 
operation that is also directed toward accountability. 

With the July 1971 reorganization of the Department of 
Public Instruction, this Section was assigned the responsi- 
bility for evaluating all Title I and Title III programs, 
selected Career Programs, and selected State Programs. 

The purpose of this paper is to describe the basic philosophy 
and the activities by which the evaluation will be accomplished. 

RATIONALE 

In the past, the roles of the local project evaluator 
and the Department of Public Instruction evaluator of a 
federally-funded or state-funded project have overlapped 
causing a confusion of purposes. Given a broad enough 
definition of evaluation which allows evaluative activities 
to be of different types, it is proposed that the activities 
of each should be interlocking and supportive. In order to 
be applicable to both, evaluation is defined as the 
systematic procedure of collecting and analyzing informa- 
tion for distribution to those involved in the decision- 
making process . The information obtained should provide a 
basis for making rational judgments. 
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In terms of the individual project at the local 
level, evaluation has come to be recognized as an integral 
part of good management. The local project personnel have the 
responsibility for the evaluation design, instrumentation, 
data collection and analysis within the scope of the proposed 
program objectives. Thus, at this level the evaluation 
process is the medium through which management gathers data 
and analyzes the process and product for its own information 
and its own report system. 



Given this, the responsibility for objectivity is 
assigned to the members of the Evaluation Staff at the 
state level whose role is Interpreted to be that of an 
evaluat or — aud i t o r . The use of an audit procedure as a 
means for evaluating education has frequently been advocated 
by critics of our present educational system. Governor 
Peterson has again spoken for this group when he stated 
that "the evaluation or audit process" is necessary in 
order to obtain "an honest accounting of what has happened 
to children’s attitudes, skills, and the level of knowledge 
in relation to locally-established objectives and goals." 

He further noted 

To be effective, the audit must be based on 
the objectives of the course or program as 
developed by the staff, students, or even the 

tives must be specific and 
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the independent auditor evaluates the results 
and files a report at an open meeting giving 
both the accomplishments versus objectives 
and recommending ways the program may be 
made more effective. C5) 

Dr. Leon Lessinger, former USOE Associate Com- 
missioner, has also stated that one of the three basic 
assumptions of accountability is to have an outside independent 
review of demonstrated student accomplishment. (6) The 
USOE in pursuing this line of thought has developed procedures 
for the educational audit. The educational auditor is to 
contribute a perspective gained from his position external to 
the structure of the project management and from his exper- 
ience with similar projects. The essential characteristics of 
the auditor’s abilities are knowledge of program design, 
operation, evaluation, and management. The USOE believes, 
if properly implemented, "program auditing could well be 
one of the more significant contributions to educational 

administration in recent years." (2) 

The educational program audit is basic to the proposed 

evaluation procedures of the Evaluation Section. The audit 

function at the state level will include a review of 

» 

1. Program management 

2. Program operation (planning, needs, objectives, 
activities and evaluation) 

3. Funding 

4. Personnel 

5. Facilities, materials, and equipment 

6. Time schedule 
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It should be noted that, in order to maintain his objectivity, 

the educational auditor is not to be considered a general 

consultant and as such cannot become part of the local or 

state management team. Furthermore: 

Designing or modifying the evaluation system 
is not a function of the auditor. Should the 
auditor note discrepancies between the project 
objectives and the proposed Instruments or 
procedures for evaluating them, he should 
communicate these discrepancies in his taudlt 
reportj. The selection of an appropriate 
alternative corrective action should always 
be a local management decision. (2) 

While these activities help to define educational 

auditing, a system needs to be devised whereby the process 

becomes operational. The five phases of the audit system 

to be employed by the Evaluation Section are the pre-audit, 

site visit, audit report, dissemination of the audit report, 

and reply. These phases may be cycled through once, twice, 

or several times depending upon the nature of the project. 

Also, for selected programs more than one educational 

auditor may be involved. A descriptive outline of each of 

the five phases follows. 



EDUCATIONAL AUDIT SYSTEM 



Phase I-Pre-Audit 

During the pre-audit stat>e, the educational auditor 
will read and become familiar with all the available infcrma- 

r 

tion on a project (including the original proposal, LEA 
project reports, and Department of Public Instruction 
administrative reports). This, of course, presupposes 
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access to pertinent documents. At this time a critique of 

the approved proposal will be written and filed. Certain 

general guidelines will be observed in reviewing the project 

proposal. These are: 

a worthwhile proposal is one which. ..uses 
appropriate, adefquate and efficient pro- 
cedures in achieving objectives; shows 
promise of success via careful planning; 
makes provision for rigorous evaluation; 
and employs reasonable cost. (3) 

This preliminary preparation will also include a 
list of questions raised during the reading of the available 
reports. These questions will be asked of the project 
director and other appropriate project personnel at the time 
of a site visit . 

Phase Il-Site Visits 

In order to obtain a realistic appraisal of a project's 
activities, it is essential that they be viewed first hand. 
Thus, the second phase will include one or more site visits. 
Since the rationale of a site visit is to gather information, 
the activities of the educational auditor wixl be questioning, 
interviewing and observing. The purpose of the visitation 
will be to learn about those features of the project which 
cannot be learned through the reading of the project proposal 
and reports; to clear up any confusion which may have resulted 
from incomplete and/or inaccurate reports; to validate the 
accuracy of objective data included in the reports; and to 
provide a basis for making comparisons of subjective per- 
ceptions included in the reports. (4) 
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Specifically, the above activities will be accomplished 
by observing class. 'oom situations using the experimental 
techniques (if any) , interviewing pupils and teachers using 
new procedures, and gathering the reactions of the adminis- 
trative personnel. Accurate description and assessment of an 
educational situation in its existing context cannot be 
accomplished by any other means. 

During the site visit, all records including the 
following should be made available to the auditor: evaluation 

progress reports, collected data, anecdotal information and 
the budget. Without these records the audit report will 

s 

only be as complete as the best available information, while 



access to them will enable the auditor to be factually correct. 
An exit interview with the project director and/or with the 
local chief school officer of schools, if it is at all possible 
to arrange for his presence, will be held to confirm the 
Information obtained by the educational auditor and for the 
auditor to note the general direction of his intended report. 



Phase III-The Audit Report 

After the site visit has been concluded, the evalua- 
tion auditor will prepare a project audit report. This will, 
in essence, be a comparison of the proposed program manage- 
ment, operation, funding, personnel, facilities, materials, 
equipment, and time schedule with the actual status of them. 
The purpose of such a report is to provide feedback to the 
project director and other responsible program personnel. If 
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correctly implemented, the audit report should assist them 
in making more rational decisions in their continual efforts 
to improve the quality of the program. (9) 

Generally, the audit report will contain descriptive 
statistics of the project, a summary status or overview, a 
comparison of the proposed and actual activities, and recom- 
mendations. The report as a whole will indicate, with support- 
ing evidence. Insofar as feasible, the extent to which the 
project is accomplishing what it was designed to accomplish. 
Thus, the auditor will assess the fulfillment or non-fulfill- 
ment of those key steps which having been carefully planned 
during the project development are being taken to meet the 
project objectives. 

Phase IV-Dissemination 

The educational audit report, after review by the 
State Program Coordinators for the program involved, will 
be sent to the local chief school officer and the local 
project director. In addition, copies of the audit are to be 
distributed immediately upon completion to members of the 
Administrative Council, DPI. A report of completed audits 
without the inclusion of the audit statement will be sent to 
the State Board of Education as a Section III agenda item. 

P hase V-Reply 

A reply to the audit report is to be made by the 
school district involved to the Director of the Division of 
Planning, Research, and Evaluation within three weeks of 
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receipt of the audit report. Upon receipt of this reply, It 
will be acknowledged by the Director of the Division or the 
person responsible for audits In general. If challenges are 
raised by local school personnel In this reply, then appro- 
priate personnel in DPI will acknowledge those challenges 
and seek a resolution of the problems created. 

CONCLUSION 

The ultimate goal of all parties in this proposed 
system is the same — assured program success, A realistic, 
objective appraisal of the educational processes is being 
demanded now by a concerned citizenry. The position taken 
in this paper is that the Evaluation Section of the Planning, 
Research, and Evaluation Division can best accomplish its 
role through the educational audit. This should result in 
an accurate account of both the strengths and weaknesses 
of existing educational programs with an objective toward 
recommendations for improvement. The final responsibility 
for decision-making changes, however, rests in the local 
school district. 
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